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ABSTRACT 

The image and effectiveness of an economics and 
business department at a medium-sized Christian liberal arts college 
were assessed, in order to assist in departmental planning for the 
future. Using goals from the extended mission statement of the 
department, two groups of graduates were surveyed, 421 recent 
graduates who had graduated between 1 and 5 years ago and 130 older 
graduates who had graduated 10 to 15 years ago. In addition, recent 
graduates were asked to pass along an anonymous questionnaire packet 
to their immediate supervisors containing a modified version of the 
graduate questionnaire asking respondents to rate the department. 
Results indicated that a few departmental areas needed increased 
emphasis but that most of the graduates who went on to graduate 
school found themselves prepared either equally or better than others 
in their graduate programs. Most agreed that they would get the same 
degree from the same school if they had to do it over. Recent 
graduates were most likely to have useable suggestions for 
departmental improvement. The economics and business department used 
the study results to develop a strategic plan for itself, to inform 
the Business Advisory Council of the results, and to persuade all 
department members of the need for change. Six references are 
included. (JB) 



ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft * 5V ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft ft * ft ft ft ft ft ft ft ft ft ft ft ft ft 

Keproauc Lions supplied oy are ztie best Lhat can be made K 

* from the original document. * 

ftftftftftftftftftftftftftftftftftftftftVcftftftftftftftftftftftftftrtftftftftftftftftftftft 



An Assessment of an 
Economics and Business Department: 
Surveys of Graduates and their Supervisors 



Prepared for Presentation 
at the 
32nd Annual Forum 
of the 

Association for Institutional Research 
Atlanta, Georgia 

iviav iv-ld, iyyA 



N 
Q 

i: 



Bv 

AnnW. Annis, 
Assistant Director 
and 

Rodger R. Rice, 
Director 



"PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 

AIR 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



U.S. DEPARTMENT OF EDUCATION 
OH»ce of Educational Reseaichand improvemen? 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

y^This document has been reproduced as 
received from the person or organization 
originating it 

O Minor changes have been made to improve 
reproduction quality 

• Points of view or opinions stated :n this docu- 
ment do not necessarily represent official 
OERI posrtton or policy 



Calvin College Social Research Center 
3201 Burton Street SE 
Grand Rapids, MI 49546 
(616) 957-6420 or 957-6241 



ERIC 



BEST COPY AVAILABLE 



/ ^ i 

for Management Research, Policy Analysis, and Planning 



This paper was presented at the Thirty-Second 
Annual Forum of the Association for Institutional 
Research held at the Atlanta Hilton & Towers, 
Atlanta, Georgia, May 10-13, 1992. This paper 
was reviewed by the AIR Forum Publications 
Committee and was judged to be of high quality 
and of interest to others concerned with the 
research of higher education. It has therefore 
been selected to be included in the ERIC Collection 
of Forum Papers. 

Jean Endo 

Chair and Editor 

Forum Publications 

Editorial Advisory Committee 



ABSTRACT 



Goals from the extended mission statement of an economics and business department at a 
medium-sized Christian Liberal arts college were used to assess the department's image and 
effectiveness and help it plan for the future. Two groups of graduates were surveyed — recent 
graduates, those who had graduated between one and five years ago, and older graduates, those who 
had graduated 10 to 15 years ago. In addition, recent graduates were asked to pass along an anonymous- 
questionnaire packet to their immediate supervisors; all follow-up was conducted through the 
graduates. This paper reviews the project methodology, gives a summary of selected findings, shows 
how the department is using the results, and critiques the project with replication in mind. 



Introduction to the Project 

In April 1991, the economics and business department of a medium-sized (about 4000 students, 
mostly undergraduate) Christian liberal arts college asked the Social Research Center to conduct an 
image survey. The External Affairs Task Group of the department had formulated this statement of 
purpose for the survey: 

To conduct a strategic survey of our primary constituent groups in order to assess the 
strengths, weaknesses, opportunities and threats of the Economics and Business 
Program. These constituent groups should include alumni who majored in business 
or economics, alumni who employ/hire department majors, non-alumni employers of 
department majors, faculty, current students (all majors), current business majors. 
The college's limited institutional research budget precluded surveying all groups desired by the Task 
Group, but from this beginning a more workable project was defined. In the end, the department decided 
to assess its preparation of graduates, either to work in the business fie»*> w f o attend graduate school; 
to gather suggestions from graduates and their immediate supervisors for improving the department; 
and to assess the perceived image and standing of the department in relationship to its educational 
competitors for business majors. 

A Business Advisory Council made up of business men and women had been formed to help the 
department in its planning and to give contact with the world that employs its graduates. After 
consultation with both department members and the Business Advisory Council, the Social Research 
Center agreed to do a study which would assess the image and effectiveness of the department using 
three surveys: 1) one of older graduates, those who graduated 10 to 15 years ago; 2) a second of recent 
graduates, those who graduated one to five years ago; and 3) a survey of the immediate supervisors of 
the recent graduates. Eligible graduates were those completing any of five majors — business, economics, 
economics-mathematics, accounting, or business and social science group — according to the Alumni 
Relations Office's records. To keep costs down, it was decided to survey only graduates living in the 
United States. 
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This paper will review the project methodology, give a summary of selected findings, show . 
how the department is using the results, and critique the project with replication in mind. 

Literature Reuiem 

It is relatively easy to select a sample of graduates, once sample parameters have been set, 
assuming the college has a computer file of alumni which lists college major. However, methodologies 
for surveying employers or supervisors of graduates vary greatly. Knoblauch & German (1989) 
surveyed, with an 18% response, companies and agencies which had recruited at their school during the 
last five years . Maui Community College sent a questionnaire to business firms, governmental agencies, 
and non-profit organizations in the community (Kong, 1976). Lucas (1984) asked alumni in a telephone 
interview for permission to contact employers and received responses from 72% of the employers of the 
78% of graduates who gave permission (56% response rate). Kapraun & Nienkamp (1985) asked 
employers' names and addresses in a graduate follow-up survey and surveyed only those employers of 
graduates working in their major field of study; 39% of the 221 employers thus identified responded. At 
the University of Tennessee-Knoxville, graduates were mailed alumni surveys which then determined 
whether they were employed. All employed graduates were then sent a letter detailing the purpose of 
an employer survey and asking permission to contact their immediate supervisors as well as the name, 
address, job title and a description of duties of that supervisor. Of those respondents working full-time, 
21% gave permission to survey their employers. Employer questionnaires and a copy of the graduate's 
permission were sent to the employers named and 93% of them responded, for an overall response rate of 
19.5% — based on the initial graduate sample (Banta, Phillippi & Lyons, 1991). 

Most supervisor surveys looked for qualities employers felt most important in a graduate or 
asked supervisors to evaluate the individual graduates. Rather than evaluate individual graduates, 
the department wanted to determine how well its preparation of graduates was perceived. We decided 
to have supervisors evaluate the department and its preparation of graduates, based on experiences 
with the graduates. 



Questionnaire Oeuelopment and Design 

The department had previously established goals for its majors in an extended mission 
statement. After brainstorming with the Business Advisory Council and department members to help 
determine which goals were most important to evaluate, the Social Research Center developed items 
to test how well department graduates and their supervisors perceived the department was reaching 
these goals. The questionnaire was designed to discover whether graduates and supervisors felt the 
department goals were important in the "real world" and how well they felt the department and the 
school had prepared the graduates. 

Three separate five-part questionnaires (one for each group of graduates and one for 
supervisors) were designed and pretested. The seven-page final versions of the questionnaire for the 
two groups of graduates were nearly the same; one last section was added to the recent graduates' 
questionnaire asking whether the graduates were willing to pass on the supervisor's packet and for 
some information about their supervisors. The supervisors' questionnaire was similar to the graduates', 
eliminating items requiring knowledge about the department they likely did not have. The structure of 
the questionnaires was the same: five parts, as follows. Part I asked respondents about their 
impressions of the department. Included were a set of semantic differential items and a set of Likert- 
scale items. Part II, the primary evaluation tool, was designed for respondents to evaluate the 
preparation of graduates by the department; it asked how important it was that the department 
develop certain qualities in its graduates and how well the college prepared the graduate in each area. 
Thirty-four qualities which related directly to departmental objectives were grouped into categories of 
values, knowledge, and skills. Part III asked for suggestions for improving the department. It included 
a set of closed-ended questions and one open-ended question about possible directions for the 
department. Part IV asked respondents to compare the college's preparation with that of its 
competitors, including (for graduates only) whether they would get the same degree and get it from the 
same college if they had it to do over. Part V of the questionnaire requested demographics and other 
background information about respondents. 



Data Collection Procedures 

Samples 

The Social Research Center obtained from the Alumni Relations Office two samples of 
randomly-selected graduates in the five designated majors who were still alive, living in the United 
States, and who had active addresses: 1) 800 recent graduates and 2) 200 older graduates. The sample 
of recent graduates included nearly all (98%) who were eligible, and the sample of older graduates was 
62% of all eligibles. These samples included the newly-graduated as well as those who had been in 
the workplace for awhile. Recent graduates (those who had graduated from one to five years ago) 
were asked to pass on a separate survey packet to their immediate supervisor. Based on the literature 
review, we projected a low percentage (about 20%) of recent graduates who would pass on the 
supervisor's packet. For this reason, we set the recent graduates' sample larger to generate a usable 
response (at least 100 completed questionnaires) from supervisors. The older graduates' sample, which 
was expected to yield a higher response rate (65%), was set smaller due to budget constraints. 

Mailings 

Both graduate groups were mailed packets, each consisting of a personalized cover letter signed 
by the department chairperson, questionnaire, and business reply envelope. The graduate surveys were 
confidential, not anonymous, meaning that each questionnaire contained an identification number 
solely for the purpose of removing a name from the follow-up list, but that responses would never 
identify any individual respondent Two weeks after the initial mailing, all nonrespondents were sent 
a reminder postcard; after an additional two \veeks, all nonrespondents were sent a second complete 
packet. Thank you cards were sent upon receipt of the completed questionnaire. First-class postage 
stamps were used on all mailed pieces. 

The packet sent to recent graduates was slightly larger because it also contained a second 
envelope with the words "Immediate Supervisor's Packet" typed on the front. Enclosed in this 
envelope were a generic cover letter to the supervisor (signed by the department chairperson), an 
anonymous questionnaire (with no identification number), a stamped anonymous reply card containing 
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the graduate's identification number, and a business-reply envelope for returning the completed 
questionnaire. The graduate received a consent card to sign and insert into the immediate supervisor 
packet before passing it on. If any chose not to ask their supervisor to participate, they were instructed 
to indicate this on the consent card and return it to the Social Research Center. Since the immediate 
supervisor pa Jcet was unsealed, the graduates could see that questionnaire before giving it to their 
supervisors. CHir intention here was to reassure them that the questionnaire evaluated the department, 
not them, and should not be seen as a personal threat. 

Immediate supervisors of recent graduates who did pass on the packet were asked in the cover 
letter to complete a questionnaire similar to that mailed to the graduate. Again, the questionnaire 
evaluated the department that had prepared the graduate, and not the individual graduate, which 
the cover letter explained. 

Follow up of supervisors who had not returned anonymous reply cards was conducted through 
the graduates. (Although we asked for the employer's name and address, that information was not 
utilized.) Three different follow-up packets were sent out: one where neither graduate nor supervisor 
had responded, another where graduate had responded and said s/he would pass along the packet but 
supervisor had not returned an anonymous reply card, and the third where the supervisor had 
responded but the graduate had not. 
Response Rates 

A summary of responses to the three surveys is shown in Table 1. Response rate is the number of 
completed questionnaires divided by the number in the total sample (less ineligibles), figured as a 
percentage. Completed questionnaires were received from 68.4% of eligible older graduates and 52.9% 
of eligible recent graduates. Ineligibles were mostly graduates whose departmental major was 
misclassified in the Alumni Relations Office files. 

When asked, "Will you give your immediate supervisor the enclosed packet?/' 165 (39.2%) of 
the 421 recent graduate respondents said yes. Of those 165 supervisors, 117 (70.9%) returned a 
completed questionnaire. Of the 421 responding recent graduates, 177 (42.0%) refused to pass on the 



supervisor's packet, and 70 (16.6%) were ineligible because they had no immediate supervisor. The 
nine nonrespondents to that question accounted for 2.1%. Reasons given for ineligibility were that they 



had no supervisor, were self-employed, homemakers, students, or unemployed. Most refusals gave no 
reason. Response rate to the survey of supervisors was 16.1% (117 divided by total sample of 800 less 
ineligibles). 



Table 1: Response to Three Surveys 






Recent Grads 


Older Grads 


Supervisors 




(N=800) 


(N=200) 


(N=800) 


Completed questionnaires 


421 


130 


117 


Refusal 


4 


0 




KcIUSal lO pass un supervisor d palKci 






177 


Ineligibles 


4 


10 


4 


Ineligibles, no supervisor 






70 


Unable to contact (return to sender, etc.) 


3 


0 


3 


Nonrespondents 


368 


60 


•445- 


Response rate (based on total sample 








less ineligibles) 


52.9% 


68.4% 


16.1% 



Data Processing 

Lading 

Each completed graduate questionnaire that was returned was recorded to avert further follow- 
up, assigned a new identification number, and its responses were coded. Each completed supervisor 
questionnaire returned was numbered and coded. Recent graduates were given credit for returned 



supervisor questionnaires when an anonymous reply card arrived with the graduate's original 
identification number written on it. 

Our quality-check system requires that on average one of every three questionnaires be check- 
coded; if problems are found, more may be checked, if none, it may go as low as one of every five. In 
these surveys, every fourth or fifth case was checked, depending on the accuracy of the original coder. 
Data were then keypunched into the college's Prime 750 computer, and the entire data files verified 
against the original code sheets. SPSSx was used to tabulate the data. 
Comments and Open-Ended Responses 

In the second phase of data processing, all open-ended responses and comments from each 
questionnaire were typed verbatim, in question order, into a word-processing file. Once completed, the 
file was spell-checked, then proofread before pagination. Open-ended responses were not coded. These 
typed responses and comments, along with statistical results, were given to the department as a report. 
The department did its own data analysis. 

Selected Results 

Each questionnaire began with a semantic differential exercise, pairs of opposite terms 
representing extreme values on a five-point scale by which respondents could describe their impressions 
of the department. The data reveal differences in the image of the department each group possesses 
and, by comparison, changes in the department image between recent and older groups of graduates. 
However, the mean scores showed only slight differences. We then aggregated those who circled 
either 1 or 2 as preferring the left-hand term in each pair and those circling 4 or 5 as the right-hand 
term and then looked at all terms that scored over 50%. Recent graduates showed more consensus in 
their impressions than the other groups. They saw the department as Christ-centered (83.4%), theory- 
driven (703%), current (63,6%), rigorous (62.2%), relevant (57.0%) and successful (57.0%). The older 
graduates saw the department as theory-driven (77.7%), Christ-centered (73.1%), economics-oriented 
(70.8%), and r.gorous (662%). Because of the department's emphasis on business in the last 10 years, 
economics-oriented scored lower among recent graduates. Supervisors saw the department as Christ- 
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centered (52.7%), rigorous (50.9%) and practical (50.0%); however there is little consensus among them 
as to department image. (Supervisors were given only 10 of the 17 pairs to which graduates responded; 
these included only the first and fourth ranked terms from recent graduates and all but the first term 
rated by older graduates.) All three groups saw the department as Christ-centered and rigorous. 

The data from Part II of the questionnaires, which asked respondents to evaluate the 
department's preparation of graduates, were intended to do two things: 1) evaluate the departmenf s 
goals for majors and 2) show changes which have occurred between times of graduation of the two 
groups as well as areas which are in need of improvement. The results we present here relate to the 
second purpose and are selective. 

We crosstabbed how important respondents saw each quality the department was trying to 
develop in its graduates with how well they thought the college had prepared them, collapsing "very 
important" and "important" to one category, "important," and "very inadequate" and "inadequate" to 
one category, "inadequate." The percentage who saw each quality as important but felt inadequately 
prepared was compared for older graduates and recent graduates. Table 2 examines those comparisons 
where the greatest changes seem to have occurred — those for which there was at least a ten- 
percentage-point difference — between older graduates and recent graduates. For no quality did recent 
graduates feel more inadequately trained than their older col .iterparts. Only one of the eight qualities 
listed under "Values" in the questionnaire appears in the list of Table 2, whereas three of eight 
"Knowledge" qualities and seven of 18 "Skills" qualities are in the table. The department 
traditionally has stressed values and seems to have improved in knowledge and skills preparation of 
its graduates. 



9 



Table 2: Percentage of Respondents Who Said Selected Quality Was Important But 


Preparation Was Inadequate 






Quality Recent Grads 


Older Grads 


Values 






Healthy balance of work, family, worship, leisure 


23.6 


35.2 


Knowledge 






Management frameworks within which a business plans and operates 


13.9 


30.5 


Concepts and terminology of marketing 


10.8 


33.6 


Knowledge for long-term career flexibility 


31.4 


44.7 


Skills 






Skills in creating and using computerized business information 


45.5 


68.5 


Skills in interpersonal communication 


24.1 


i3.3 


Skills in public speaking for work-related settings 


40.7 


51.3 


Skills in leading business co-workers in analysis and decision ms ing 


20.6 


30.0 


Capacity to adjust to new job demands 


15.4 


31.6 


Skills in handling several tasks at once 


11.2 


28.4 


Skills in planning projects 


14.5 


29.0 



Just as previous studies have indicated, we found that supervisors generally thought the 
college had prepared their employees adequately, although many were uncertain, too. The highest 
levels of inadequacy reported by supervisors were for skills in public speaking (14.6%) and skills in 
written communication (12.6%), but these are relatively small percentages. In many instances, 
supervisors of the department's recent graduates did not know whether to attribute their employees' 
skills, knowledge and values to the school which trained them. One supervisor stated, "I don't know 
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whether my employee got these skills at [the college] or in a previous position, so these ratings are for. 
the employee's current skills, not necessarily what they got from [the college]/' 

Qualities for which more than one third of recent graduates said they were inadequately 
trained were skills in creating and using computerized business information (455%), skills in public 
speaking for work-related settings (40.7%), and ability to work with people of diverse backgrounds 
(37.2%). Older graduates felt most inadequately prepared in skills in creating and using computerized 
business information (64.7%), skills in public speaking for work-related settings (49.2%), knowledge for 
long-term career flexibility (415%), skills in interpersonal communication (40.8%), and ability to work 
with people of diverse backgrounds (37.7%), 

When asked in Part III for suggestions for improving the department, most of each surveyed 
group felt the department should expand its relationships with the business community (r, 94,1%; o, 
90.8%; s, 89.5%), More than 50% of graduates either agreed or strongly agreed with almost all 
suggestion statements. However, about half (r, 47%; o, 55%) of all graduates were uncertain whether 
the department should recruit more ethnic minority students into the business major. The problem with 
the minority question stems from the quandary of "how much is more?" Over half (r, 65.3%; o, 56.9%) 
disagreed or strongly disagreed that the department should reduce the number of liberal arts core 
courses in all its business majors. In addition, only 392% of older graduates, in comparison to 54.8% of 
recent graduates and 51.7% of supervisors, felt the department should internationalize its business 
programs. Supervisors were slightly more likely to believe that the department should develop a 
cooperative education program for business students (r, 58.7%; o, 54.6%; s, 64.1%). All respondents were 
asked to list their first, second and third most important of the given suggestions; all three groups saw 
the same three areas as most important: some applied business experience should be required of every 
business major before graduation (r, 603%; o, 52.3%; s, 573%), the department should teach more 
computer application (r, 473%; o, 44.6%, s, 38.5%), and the department should expand its relationships 
with the business community (r, 45.4%; o, 43.1%; s, 43.6%). 

Part IV of the questionnaires involved comparing this school's preparation of graduates to that 
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o'f competitors. When recent graduates were asked, "Where would you find the educational program 
that would prepare students best for the following areas of a business career today?/' their modal 
responses were the college for values (89.2%) and knowledge (45.2%), and major university (40.8%) for 
skills. Older graduates' responses were similar with the college for values (86.2%) and knowledge 
(37.7%), and major university (36.2%) for skills. Supervisors, however, rated the college high for 
values (71.1%), major university for knowledge (42.1%) and for skills (34.3%). All three groups of 
respondents gave the college lower marks than major university and private business college in the 
skills area; less than 4% of supervisors rated the college best for skill preparation. 

Who were the respondents? Although many recent graduates were female (43.2%), the 
majority were male (56.8%). They were married (58.0%), with no children (78.6%). They do not 
volunteer their business and economics skills (55.6%). A majority majored in business (50.4%), and have 
not worked on a graduate degree (74.3%) but most who have gone to graduate school found themselves 
prepared either equal or superior to others in their graduate programs (925%). They have had one or 
two jobs since graduation (75.8%) and were working full-time (84.8%) in a job somewhat or very related 
to their major (805%). The majority worked in the service or knowledge industries (50.4%). Two-fifths 
(41.6%) earned $25,000 to $49,999 a year. 

More of the older graduates were male (73.8%), married (86.9%), with two or three children 
(63.1%). Almost two thirds (63.1%) volunteer their business and economic skills at church. They 
majored in business (50.0%), and have not worked on a graduate degree (58.5%). Most who did go to 
graduate school found themselves prepared either equal or superior to others in their graduate 
programs (94.2%). They have had between one and three jobs since graduation (653%) and were 
working full-time (823%) in a job somewhat or very related to their major (80.0%). The highest 
percentage worked in the service or knowledge industries (47.7%). One third (32.8%) earned $25,000 to 
$49,999 a year and 44.7% earned more. 

Supervisors were mostly in their 30s or 40s (675%), and were not graduates of the college 
(71.1%). They worked in for-profit organizations (85.1%) of 500 or fewer employees (69.4%) in the state 
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of Michigan (68.4%). 

Uses by Department 

The economics and business department is using the results of this project in three ways. First, it 
is developing a strategic plan for itself. The survey data have helped them find a common basis of 
understanding of where they are educationally at this time, so they can proceed with planning for the 
future. The results have shown the department ways in which they can improve their interactions 
with the business community and train their students better. They are also working to build the public 
awareness and image of their program in an attempt to improve the department's market share of the 
traditional student population. 

Second, the results were presented to the Business Advisory Council so it could be fully informed 
and help with the department's strategic plan. In part, members of this committee keep the 
department in touch with the business world. 

Third, results have been distributed to all department members to persuade them of the need 
for change. Following good assessment principles, the department faculty must have ownership of any 
proposed changes and not view those changes as personally threatening. By assessing the department 
as a whole, faculty are able to see curricular strengths and weaknesses. 

The department views this survey project as a base and intends to replicate it in five years after 
implementation of its strategic plan. Replication will show if the department is moving in the desired 
direction, producing graduates better prepared to work in their chosen economics or business fields. 

Critique and Replication Suggestions 

Here we shall reflect on the innovative procedures employed in our study. We shall also 
consider the questions, "What could we have done to make the project better?" and "What would we do 
differently, if we replicate?" 
Graduate Populations Surueged 

We surveyed recent and older graduates both to detect change over time and to gain a broader 
view of the department's image. Our impression is that most older graduates surveyed have not kept 
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up with the department and are unaware of changes which have been made* It is important to note, as 
well, that older graduates may not remember how well their college program prepared them, thus 
accounting for the sometimes high percentages of uncertain responses. Of the 11 qualities listed in Table 
3, in eight cases 33.1% or more of older graduates were uncertain, but for only two qualities was this true 
for recent graduates. It probably is wiser to confine research for evaluating how well a department 
prepares its graduates to those who have more recently graduated. Furthermore, the higher 
uncertainty responses among older graduates produce difficulties for measuring changes over time. 



Table 3: A Comparison of Percentages of Recent and Older Graduates Regarding their Uncertainty 


Towards Their College's Preparation of Them 








Recent 


Older 




Grads 


Grads 


Oualitv 


(N=421) 


(N=130) 


Ability to recognize business and economic issues requiring moral 






sensitivity 


13. 1 


23 .0 


Ability to make sound moral decisions about workplace practices 


15.4 


24.6 


Problem-solving skills 


15.9 


26.9 


Ability to work as a team member 


17.8 


33.1 


Skills in handling several tasks at once 


19.0 


33.1 


Skills in planning projects 


24.2 


33.8 


Ability to work with people of diverse backgrounds 


24.7 


36.2 


Insight into appropriate Christian leadership styles 


24.9 


36.2 


Understanding religious perspectives underlying commonly-used 






business and economic theories 


28.7 


40.8 


Skills in leading business co-workers in analysis and decision making 


34.7 


43.1 


Capacity to find new approaches to adapt to a changing environment 


35.4 


44.6 
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Superuisor Selection Method 

In part because of budget and time constraints, we chose the method of asking recent graduates 
to pass along the immediate supervisor's packet In our study, 39.2% of recent graduate respondents 
agreed to give their supervisors the questionnaire packet, and 70.9% of those supervisors responded (but 
this was only 16.1% of all possible supervisors). (When University of Tennessee-Knoxville sent their 
questionnaires directly to the employer after receiving permission over the phone, they received an 
overall response rate of 19.5%.) Our pass-along rate was higher than we anticipated, based on the 
literature, but the response rate from supervisors was lower. There is also the problem of 
representativeness of the supervisor sample. Although our sample of recent graduates was nearly all 
eligible graduates, we could not control which supervisors received a questionnaire. Of recent graduates 
who did pass on the questionnaire, only 6 (3*7%) said their job was unrelated to their major. This 
suggests that taking the time and money to eliminate supervisors of graduates who were not working in 
a job related to their major, as some other studies have done, would have been unnecessary because 
graduates such as those chose not to pass on the questionnaire. 

Some paris of the procedure we used were unnecessary, we think* We asked graduates for the 
name and address of their immediate supervisors, and then did not use that information, but followed 
up through the graduates. Perhaps not asking for that information would have yielded a higher pass- 
along rate. Additionally, we asked the recent graduates in their questionnaires whether they would 
pass along the supervisor's packet, and then also instructed them to return their consent card to us if 
they were not doing so. Having respondents return the consent card to us was redundant and 
unnecessary. These procedures need to be streamlined. If we are not going to use the information, it 
should not be asked* 

How useable, though, was the information we gathered from supervisors? Was it worthwhile 
to include this component in our study? Supervisors tended either to give the college high marks or to 
say they were uncertain. Only seldom did they give low marks. This may be because of a reluctance to 
respond negatively or because only the more competent graduates were willing to pass along the 
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supervisor's packet. In this survey, supervisors' responses were not seen as a highly reliable assessment 
of the program. 
Departmental Evaluation 

Unlike many previous employer/ supervisor surveys, which looked at qualities employers look 
for in their hires, our project was also designed to assess the goals of the department and its 
preparation of graduates. Just as a department's faculty may feel threatened by an assessment of their 
specific courses, we felt the graduates might feel threatened by an assessment of their performance. 
Besides, we wanted to determine how well the department was accomplishing its goals in general, not 
necessarily how well any given individual was doing. 

For some supervisors, their only contact with the college had been the one current employee. 
When asked to evaluate the department, not the employee, some found the questions difficult to 
answer. The difficulty of the task probably contributed to a response rate lower than we anticipated. 
One supervisor suggested that a capsule summary of the school's program might have helped. They 
could have more easily evaluated their employee, but had that been the case probably fewer graduates 
would have asked their supervisors to participate — a trade-off in response rates? In any case, if it is to 
be used at all, the supervisor questionnaire musi be modified with the respondents in mind, asking 
realistic questions which they can answer. This may require asking questions different from those 
asked of the graduates, precluding the opportunity to compare responses, an option which the 
department felt it needed. We could have asked the supervisor if their employee's skills were as good 
as those of employees from other schools, and in what respect. They could answer that type of question. 

In addition, the survey goals must be refined so the questionnaires can be shortened. Initially 
the Business Advisory Council wanted a two-page employer questionnaire, but they wanted to measure 
too much to keep it that short. Shorter instruments for all groups would probably help increase response 
rates. 

As many of the supervisors do not know the program, information from graduates is much more 
valuable — they know the program better and they know how well it has prepared them to work in 
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their chosen field. 

Conclusion 

In this project, the economics and business department wanted to assess its preparation of 
graduates, either for work or to attend graduate school; to gather suggestions from graduates and their 
immediate supervisors for improving the department; and to assess the perceived image and standing of 
the department in relation to its educational competitors for business majors. 

The questionnaires evaluated both departmental goals and preparation of the graduate in 
those areas. A few areas were singled out as needing increased emphasis. Most of the graduates who 
went on to graduate school found themselves prepared either equally or superior to others in their 
graduate programs, and would get the same degree from the same school if they had to do it over. 
Recent graduates were most likely to have useable suggestions for departmental improvement; most 
older graduates and supervisors were unfamiliar with the current state of departmental offerings. 

The surveys gave the department some very useable information. Next time, however, serious 
thought will probably be given to surveying only recent graduates. They are most knowledgeable about 
the department's current training and practices. 
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